Supplementary material for Forward-LASSO with Adaptive Shrinkage

Note that νj([2S−1]) = 1 and νj(ρ) is a strictly increasing function for ρ < [S−1]−1. The proof of Theorem 2 in Wainwright (2009) establishes that for each value of the tuning parameter λ the necessary condition for the signed support recovery is |νj + Z̃j| ≤ 1 with Z̃j = λ−1XT j ΠX⊥ K ( ). Note that Zj follows a non-degenerate zeromean gaussian distribution for S < n, thus the probability of the above condition is at most 1/2. When S ≥ n the Lasso automatically fails, because none of its estimators can have more than n− 1 nonzero coefficients.